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INTRODUCTION: _ 

This  project  aims  to  enhance  the  capacity  of  the  Air  Force  (AF)  to  reduce  death,  injury, 
and  degraded  force  readiness  via  reduction  of  the  prevalence  and  impact  of  family  maltreatment, 
suicidality,  and  alcohol/drug  problems.  Managing  risk  and  increasing  resilience  in  military 
human  resources  (i.e.,  “Force  Health  Protection”)  is  a  top  priority  for  DoD  and  Armed  Forces 
leadership.  The  objective  of  this  study  is  to  enhance  the  AF’s  current  prevention  delivery  (known 
as  the  Integrated  Delivery  System;  IDS)  infrastructure  through  (a)  the  development  and 
validation  of  a  infonnation  system  needed  to  direct  prevention  efforts  more  effectively  and 
efficiently;  (b)  the  adoption  of  a  prevention-science-based  approach;  and  (c)  the  evaluation  of  its 
effectiveness.  When  funded,  the  proposed  project  was  broken  into  two  phases.  This  first  phase  is 
a  demonstration  project  on  which  to  build  a  randomized  trial.  This  project  is  meeting  the 
objectives  by:  (a)  pilot  testing  the  development  of  an  innovative  surveillance  system  and 
validating  its  accuracy  (at  4  AF  bases)  for  family  maltreatment,  suicidality,  and  problematic 
alcohol  and  drug  use,  and  (b)  pilot  testing  the  creation  of  an  enhanced  IDS  by  training 
community  leaders  in  prevention-science-based  intervention  methodology  and  testing  the  impact 
on  factors  that  are  prerequisites  for  effective  community  prevention  initiatives  and  on  targeted 
outcomes. 

BODY: 


Year  2  —  Continuing  Activities 

Task  1 :  Provide  technical  assistance  to  IDS  teams  at  pilot  bases  in 
implementing  action  plans;  systematically  monitor  impact,  and  adjust 
implementations  accordingly 

Revise  orientation  and  training  materials/guides  to  prevention  interventions 

We  have  continued  to  revise  both  the  orientation  materials  and  the  guides.  We  have 
conducted  literature  searches  to  keep  Guidebook  to  Activities  That  Work  current,  with  a 
particular  focus  on  activities  that  are  easily  disseminable  to  large  groups.  We  have  created 
Guidebook  to  Evaluating  Activities  That  Work  to  help  bases  with  evaluation  planning. 

Provide  technical  assistance  and  briefings;  monitor  impact  and  adjust  system  as 

needed 

In  the  past  year,  we  have  continued  to  provide  technical  assistance  in  the  form  of  weekly 
calls  with  IDS  Chairs  and,  as  needed,  subcommittees.  All  bases  have  been  offered  the 
opportunity  for  additional  on-site  consultation.  Additional  phone  consultation  on  evaluation  has 
occurred  routinely,  with  our  staff  providing  measures,  data  collections  systems,  and  protocols  for 
use  by  the  bases.  Bases  have  continued  to  face  challenges  in  the  implementation  of  activities,  but 
have  also  remained  engaged.  Progress  update  briefings  have  been  held  at  participating  bases  and 
at  meetings  of  MAJCOM  IDS  Chairs  and  Behavioral  Health  representatives,  as  well  as  at  AF- 
IDS  meetings. 
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Wave  1  Bases  (Tyndall  AFB,  Barksdale  AFB,  Shaw  AFB) 

The  primary  purpose  of  this  phase  of  NORTH  STAR  was  to  pilot  test  materials  and 
procedures  before  conducting  a  full,  randomized  controlled  trial.  Because  each  base  has  a 
prevention  board  (the  Integrated  Delivery  System,  or  IDS),  we  believed  that  we  would  be 
helping  IDSs  substitute  more  effective  and  targeted  activities  for  the  activities  they  were  already 
doing.  What  we  learned,  however,  is  IDSs’  efforts  typically  fall  far  short  of  the  planning  and 
activities  mandated  in  AF  instructions.  This  lesson  was  a  critical  one  to  learn  in  the  pilot  phase  of 
NORTH  STAR,  as  it  has  allowed  us  to  take  corrective  actions  at  the  Wave  2  base  and  to  prepare 
better  for  the  upcoming  controlled  randomized  trial. 

We  continue  to  provide  technical  assistance  at  the  three  Wave  1  bases;  however  NORTH 
STAR  activities  have  only  achieved  partial  implementation  at  these  bases.  IDS  team  members 
were  eager,  engaged  and  committed  in  development  and  implementation  of  NORTH  STAR 
action  plans.  Over  time  their  activities  in  support  of  the  action  plans  decreased.  We  identified 
consistent  challenges  for  the  Wave  1  bases  including:  1)  turnover  in  IDS  Chair  with  inadequate 
handoff  /  succession  planning;  2)  lack  of  senior  leadership  ownership  or  buy-in;  and  3)  denial  of 
funding  request  by  IDS  teams  for  new  prevention  activities.  Based  on  these  findings  we 
identified  a  number  of  lessons  learned  targeted  at  improving  AF  institutional  supports  and 
rewards  to  sustain  community-wide  prevention  activities. 

Wave  1  Lessons  Learned 

•  Importance  of  senior  leadership  buy-in  and  involvement 

o  Understand  the  importance  of  community  prevention 
o  Sense  of  ownership  IDS  interventions  for  their  communities 

•  Shared  vision  /  agreement  by  IDS 

o  IDS  participants  work  together  on  mutually  developed  plan 

•  Leadership  support  to  IDS 

o  Senior  leader  appointed  as  liaison  to  IDS 
o  IDS  activities  supported  by  supervisors  /  commanders 

•  IDS  accountability  to  leadership 

o  Quarterly  metric  &  progress  briefings  to  leadership  meetings  vs.  ‘activity’ 

•  Base  leadership  accountability  to  major  command 

o  Quarterly  metric  &  progress  briefings  to  major  command 

•  Resources  required  for  success 

Wave  2  Base  (Kadena  AB) 

We  utilized  the  lessons  learned  from  Wave  1  to  make  changes  in  our  approach  to  Kadena. 
Although  our  training  remained  essentially  unchanged  we  invested  significant  pre-training 
efforts  to  increase  leadership  buy-in  and  to  ensure  that  leadership  had  accurate  expectations 
about  requirements  for  supporting  its  IDS  team  in  successfully  implementing  community-wide 
prevention  activities. 

Differences  between  Wave  1  and  Wave  2 

o  Direct  wing  leadership  involvement  and  buy-in 
o  Senior  leader  liaison  between  senior  leadership  council  and  IDS 
o  IDS  quarterly  metrics  briefings  to  senior  leadership  council 
o  Major  command  participation  and  oversight 


5 


The  Kadena  CAIB  approved  the  IDS  recommendation  to  implement  three  NORTH 
STAR  prevention  based  activities.  One  of  these  was  implemented  shortly  following  the  training. 
A  second  has  been  implemented  on  a  partial  basis.  The  third  was  fully-funded  by  the  CAIB  and 
is  presently  beginning  its  roll-out  now  with  twenty  community  members  to  be  trained  in  April. 
The  IDS  continues  to  be  engaged;  and  senior  leadership,  the  CAIB  and  the  MAJCOM  have 
continued  to  provide  support  and  meaningful  oversight  of  IDS  activities. 

The  lessons  learned  from  our  experience  at  each  of  these  sites  are  of  immediate  benefit  to 
the  Air  Force.  We  identified  some  systematic  gaps  in  the  base  prevention  consortium  structure 
and  operation,  specifically  in  the  area  of  oversight,  accountability  and  leadership  involvement 
and  follow-through.  Unattended,  these  gaps  have  the  effect  of  reducing  the  effectiveness  of  the 
IDS.  The  Air  Force  expressed  interest  in  utilizing  our  findings  to  strengthen  its  IDS  and  CAIB 
processes.  We  ‘hotwashed’  our  findings  and  their  implications  with  leadership  from  the 
Community  Behavioral  Health  Division,  who  briefed  the  Air  Force  IDS  with  recommendations. 
These  recommendations  were  written  into  a  binding  AF  guidance  document  that  was  distributed 
to  major  commands  and  bases.  Ultimately,  these  new  Air  Force  efforts  to  strengthen  the  IDS  and 
CAIB  will  improve  their  ability  to  foster  change  and  will  increase  the  chances  that  the  upcoming 
NORTH  STAR  randomized  controlled  prevention  trial  will  be  successful. 

Conduct  annual  base  leadership  follow-up  assessment 

o  One-year  base  leadership  follow-up  at  Kadena  AB  is  underway 

Task  2;  Develop  and  validate  algorithms 

•  Compare  accuracy  of  using  (a)  correction  factors  on  separate  algorithms  vs.  (b)  global 
algorithms  (Months  18-21). 

We  have  completed  the  bootstrapped  confidence  intervals  for  the  22  algorithms  (which 
will  allow  their  accuracies  to  be  detennined  and  compared  using  (a)  correction  factors  on 
separate  algorithms  vs.  (b)  global  algorithms).  Confidence  intervals  are  typically  less  than 
2%  around  the  prevalence  means. 

Task  3:  Begin  planning  for  re -ad ministration  of  AF  Community  Assessment 
and  supplement  (CA+) 

The  CA+  will  be  launched  on  April  10,  2006.  We  have  been  in  at  least  weekly  contact  with 
Maj.  David  Linkh,  research  director  of  the  AF’s  Behavioral  Health  Division,  to  facilitate 
progress  on  the  CA.  We  conducted  psychometric  evaluation  of  every  scale  administered  in  the 
2003  CA  and  made  final  recommendations  to  the  AF  for  scale  modifications.  Nearly  all  were 
adopted. 

The  Air  Force  contract  for  the  2006  survey  was  finalized  in  October  2005.  Caliber  Associates 
(Fairfax,  VA)  was  chosen  to  create  the  WWW  interface  and  conduct  the  survey. 

A  major  event  affecting  the  CA+  was  the  Air  Force  Chief  of  Staffs  decision  to  field  another 
survey  at  the  same  time  as  the  CA.  The  AF  asked  that  both  surveys  go  ahead  but  not  sample 
participants  more  than  once.  This  introduced  enonnous  complications  that  required  continuous 
negotiations  to  ensure  that  the  CA+  would  get  the  sample  size  and  random  sampling  necessary. 
We  attended  a  meeting  in  Arlington,  VA  in  Oct-05,  with  Maj.  Linkh,  the  CA  contractor,  other 
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AF  representatives,  and  the  contractor  for  the  other  survey  to  negotiate  terms.  Frequent 
teleconferences  ensued.  We  attended  a  subsequent  meeting  in  Feb-06  in  Rockville,  MD  to 
review  the  sampling  plan  and  correct  significant  problems  in  it. 

Year  3  (Months  25-36) 

Task  1 :  Oversee  CA  and  supplement  survey  administration  (including 
strategies  to  increase  response  rate  at  all  AF  bases). 

We  checked  several  iterations  of  the  CA  and  worked  with  the  CA  contractor  to  ensure  that 
the  CA  would  be  carried  consistent  with  the  plan  we  had  negotiated  with  the  AF. 

Task  2:  Conduct  proposed  HLM  analyses  to  evaluate  impact  of  Enhanced 
IDS  program  at  pilot  sites 

•  Much  of  the  work  of  this  project  is  focused  on  providing  technical  assistance  to  IDS 
teams  at  pilot  bases  to  support  improved  practice  in  the  delivery  of  scientifically- 
proven  effective  interventions.  However,  the  ultimate  question  asks  if  these  are 
efforts  successful  in  bringing  about  change  in  the  prevalences  of  secretive  problems 
as  well  as  the  risk  and  protective  profiles  in  the  test  communities.  The  data  necessary 
to  answer  this  question  will  be  collected  in  the  2006  administration  of  the  CA+.  The 
completion  of  this  task  required  an  approved  extension. 

Summary 

This  project  has  experienced  delays  due  to  changes  in  the  Air  Force’s  timeline,  most 
specifically  the  one  year  delay  in  the  CA+.  Currently  work  on  the  project  is  progressing 
successfully  with  productive  outcomes,  including  findings  of  specific  use  and  application  to  the 
Air  Force  as  it  moves  forward  to  improve  its  institutional  IDS  and  base  leadership  council 
system.  We  requested  and  received  a  one  year,  no-cost  extension  in  order  that  we  may  complete 
the  originally  proposed  aims  pushed  back  by  Air  Force  delays.  Completion  of  these  tasks  is 
important,  not  only  to  adequately  answer  the  research  questions  under  study,  but  also  to  ensure  a 
smooth  transition  from  Phase  1  (pilot  project)  to  Phase  2  (randomized  controlled  prevention 
trial).  We  protected  our  allocated  funds  for  the  delayed  tasks  and  are  in  position  to  complete  this 
research  consistent  with  our  proposed  new  SOW  timeline  (reprinted  below)  which  reflects  the 
one  year  Air  Force  imposed  delay. 

KEY  RESEARCH  ACCOMPLISHMENTS: _ 

We  developed  and  cross-validated  prevalence  estimation  algorithms.  Results  are  shown 
below.  These  algorithms  will  be  used  to  match  bases  for  the  NORTH  STAR  randomized 
controlled  prevention  trial. 


_ Prevalence _ 

Problem  Actual  Estimated  Difference 


Alcohol  Abuse  6.48  5.52  0.96 

Suicidality  4.8  4.32  0.48 
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Partner  Maltreatment 
Male-to-Female 


Physical 

1.4 

1.12 

0.52 

Emotional 

6.92 

5.88 

1.08 

Female-to-Male 

Physical 

1.4 

1.12 

0.52 

Emotional 

5.88 

6.24 

0.36 

Child  Maltreatment 

Physical 

1.4 

1.12 

0.52 

Emotional 

4.26 

3.44 

0.82 

Neglect 

36.6 

31.82 

6.82 

REPORTABLE  OUTCOMES: 


Based  on  this  research  we  have  applied  for  and  received  the  following  funding  through 
the  PRMRP  FY05  announcement: 

1.  Family  Maltreatment,  Substance  Problems,  and  Suicidality:  Randomized  Prevention 

Effectiveness  Trial  (Heyman,  PI). _ 

Objective/Hypothesis:  This  study  aims  to  enhance  the  ability  of  base,  major 
command  (MAJCOM),  and  Air  Staff  IDSs  to  reduce  death,  injury,  and  degraded 
force  readiness  through  (a)  dissemination  of  base,  MAJCOM,  and  AF  prevalences  of 
secretive  problems;  (b)  provision  of  base-level  infonnation  to  identify  and  prioritize 
risk  and  protective  factors,  (c)  assistance  in  bases’  selecting  and  implementing 
empirically  supported  interventions,  and  (d)  evaluation  of  whether  prevalences  were 
lowered.  Thus,  we  hypothesize  that  NORTH  STAR  will  enhance  military  readiness 
by  reducing  the  prevalence  of  these  threats  and  by  decreasing  the  level  of  risk 
factors  and  increasing  the  level  of  protective  factors  in  test  communities. 

Specific  Aims:  Conduct  a  randomized,  controlled  prevention  trial  to  test  the 
effectiveness  of  the  prevention  science-guided  NORTH  STAR  framework  in 
reducing  targeted  risk  factors;  increasing  targeted  protective  factors;  and  reducing 
base  prevalences  of  family  maltreatment,  suicidality,  and  problematic  alcohol  and 
drug  use. 

Study  Design:  Twelve  matched  pairs  of  bases  will  volunteer  and  be  randomly 
assigned  to  either  (a)  the  NORTH  STAR  implementation  condition  or  (b)  the  control 
condition  (which  will  receive  comparable  prevalence  and  risk/protective  factor 
infonnation  from  the  2006  AF  Community  Assessment  (CA+)  but  not  receive  any 
NORTH  STAR  training,  support,  or  consultation).  At  the  12  test  and  12  control 
bases  we  expect  average  participation  (i.e.,  912  AD  members  and  349  spouses  per 
base)  in  the  CA+,  providing  us  with  excellent  statistical  power. 
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Presentations:  Dr.  Heyman  was  a  visiting  scholar  for  two  weeks  at  Griffith  University  in 
Brisbane,  Australia  to  present  colloquia  on  NORTH  STAR  and  to  consult  on  community 
prevention  (based  on  our  PRMRP-funded  experience).  The  colloquia  are  entitled: 

1 .  “Community-Based  Prevention  for  Family  Maltreatment,  Alcohol  Abuse,  Drug  Use,  And 
Suicidality”  08-Apr-05 

2.  “Engaging  Communities  in  Prevention  Activities:  Lessons  From  Work  With  The  US  Air 
Force”  08-Apr-05 

CONCLUSIONS: 


We  are  quite  encouraged  about  the  progress  made  in  the  pilot  phase.  The  pilot  bases 
appear  to  have  the  prerequisites  to  implement  effectively  a  modern  prevention  initiative.  Base 
IDS  teams  were  very  receptive  to  the  NORTH  STAR  framework  and  are  making  good  progress 
in  designing  and  implementing  empirically-supported  action  plans.  The  goals  set  out  by  PRMRP 
reviewers  for  the  first  phase  of  the  project  are  being  achieved. 
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Revised  Time  Line 


Year  2  (Months  1 3-24) 


Task  1: 

Provide  technical  assistance  to  IDS  teams  at 
pilot  bases  in  implementing  action  plans; 
systematically  monitor  impact,  and  adjust 
implementations  accordingly 

• 

Revise  orientation  and  training  materials/guides  to 
prevention  interventions 

Months  13-18 

Month  36 

• 

Provide  technical  assistance  and  briefings;  monitor 
impact  and  adjust  system  as  needed 

Months  12-24 

Month  42 

• 

Conduct  annual  base  leadership  follow-up 
assessment 

Month  24 

Month  42 

Necessaiy  to  accomplish: 

o 

Travel  to  pilot  bases  for  follow-up  consultations/ 
trainings  (as  needed);  conduct  post-  testing. 

o 

Distribute  training  materials 

Task  2: 

Develop  and  validate  algorithms 

• 

Develop  and  crossvalidate  22  algorithms 

Months  13-16 

Complete 

• 

Bootstrap  confidence  intervals  for  22  algorithms 

Months  15-18 

Complete 

• 

Compare  accuracy  of  using  (a)  correction  factors 
on  separate  algorithms  vs.  (b)  global  algorithms 

Months  18-21 

Complete 

Task  3: 

Begin  planning  for  re-administration  of  CA  and 
supplement 

• 

Devise  and  plan  survey  logistics 

Months  17-23 

Months  30-36 

• 

Secure  final  approval  at  Air  Staff,  MAJCOM,  and 
base  levels 

Complete 

Year  3 

(Months  25-36) 

Task  1: 

Oversee  CA  and  supplement  survey 
administration  (including  strategies  to  increase 
response  rate  at  all  AF  bases) 

Months  25-27 

Months  37-39 

Task  2: 

Conduct  proposed  HLM  analyses  to  evaluate 
impact  of  Enhanced  IDS  program  at  pilot  sites 

• 

Process  evaluation 

Months  28-30 

Months  40-42 

• 

Outcome  evaluation 

Months  30-33 

Months  42-45 

• 

Provide  feedback  to  IDS  teams,  provide  assistance 
in  modifications  to  action  plans 

Months  33-34 

Months  45-46 

• 

Write-up  final  reports 

Months  33-36 

Months  45-48 

li 


